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DETAILED ACTION 

1 . This communication is responsive to the Amendment filed 9/1 3/201 0. 

2. Claims 1-23 are pending in this application. Claims 1,12 and 23 are independent 
claims. In the instant Amendment claims 1 , 3-6, 8, 1 0, 1 2, 1 7, 1 9 and 23 were 
amended. This action is made Final. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-23 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Dempski (US 2006/0244677) in view of Foxlin (US 2002/0024675). 

Regarding claim 1 , Dempski discloses a system that enables a user to view a 
virtual control panel (see paragraph [0013] where it is discussed that selected data 
associated with a device is displayed on a wearable display), and 

a portable identification element carried and manipulated by the user, the 
portable identification element being configured to define a position and orientation of 
the virtual control panel (see paragraph [0013] where it is discussed that virtual data is 
displayed in accordance with visual markers associated with labeled objects), 
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Dempski does not expressly disclose a pointing object carried and manipulated 
by the user for interaction with the virtual control panel. However, Foxlin discloses a 
head-mounted virtual reality system capable of tracking a hand-mounted object relative 
to a user's head (see paragraph [0057] and [0058]). It would have been obvious to an 
artisan at the time of the invention to include Foxlin's teachings in Dempski's user 
interface in an effort to enable user to intuitively interact with applications in a virtual 
environment (see paragraphs [0004] and [0076]). 

The modified Dempski discloses a first tracking unit (see Foxlin, fig 1 , 30) 
adapted to capture data representing a position of the pointing object (see Foxlin, fig 1 , 
14; Examiner notes that user is capable of adapting the tracking unit to capture any type 
data), 

a second tracking unit adapted to capture data representing a position of the 
portable identification element (see Foxlin, fig 1, 30; Examiner notes that user is 
capable of adapting the tracking unit to capture any type data), 

a storage unit (see Dempski, fig 1 , 36, 38, 40, 42) configured to store at least one 
pre-defined graphical interface representing a control panel the device, said graphical 
interface comprising an interface for user interactions with the device (see Dempski, 
paragraph [0012] "The databases may also store data about the work environment, 
object details, current status and location of workers, etc."), 

a graphics unit configured to generate a graphical representation of the control 
panel based on said stored graphical interface (see Dempski, paragraph [0013] 
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"selecting data from a memory storage with the data being associated with one of the 
objects associated with one of the visual markers, and then displaying the data on a 
wearable display worn by the operator."), 

a registering unit configured to register said graphical representation of the 
control panel in a fixed relation to said portable identification element, based on said 
data representing the position of the portable identification element, to produce a virtual 
control panel (see Foxlin, paragraphs [0090], [0091] and [0096]), 

a display unit configured to show the user a view comprising the real world and 
the virtual control panel projected in a fixed relation to said portable identification 
element (see Dempski, paragraph [0015] where the see-through display is discussed), 
and 

an application unit configured to perform actions in response to the users 
interactions with the virtual control panel, and configured to determine which actions to 
be performed based on the position of said user controlled pointing object in relation to 
the identification element (see Foxlin, paragraph [0105] where selecting and 
manipulating objects is discussed). 

Regarding claim 2, Foxlin discloses the system is adapted to modify the 
appearance of the virtual control panel in response to interactions between the user 
controlled pointing object and the virtual control panel (see paragraph [0105] where 
selecting and manipulating objects is discussed). 
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Regarding claim 3, Dempski discloses said graphical interface is adapted to 
display data from the device and wherein the system is adapted to generate a graphical 
representation of the data and to display the data on the virtual control panel (see 
paragraph [0013] "selecting data from a memory storage with the data being associated 
with one of the objects associated with one of the visual markers, and then displaying 
the data on a wearable display worn by the operator."). 

Regarding claim 4, Foxlin discloses said user controlled pointing object is a 
handheld pointing device or a body part of the user (see paragraphs [0007] and [0008]). 

Regarding claim 5, Dempski discloses the storage unit is adapted to store a 
plurality of graphical interfaces, each representing a control panel of a particular device, 
wherein the system is adapted to generate and display a plurality of graphical 
representations of control panels for different devices based on said stored graphical 
interfaces of the devices (see paragraph [0013] "selecting data from a memory storage 
with the data being associated with one of the objects associated with one of the visual 
markers, and then displaying the data on a wearable display worn by the operator."), 
and wherein the system further comprises: an identification unit configured to identify 
which of the stored control panels to be displayed (see paragraph [0016] where the 
unique identifiers associated with the markers is discussed). 

Regarding claim 6, Dempski discloses said identification unit comprises a 
recognition unit configured to recognize and identify devices in the environment of the 
user, and wherein the system is adapted to determine which of the stored control panels 
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to be displayed based on which of the devices is identified (see paragraph [0016] where 
the unique identifiers associated with the markers is discussed). 

Regarding claim 7, Dempski discloses said recognition unit is adapted to 
recognize and identify unique identification markings on the devices (see paragraph 
[0016] where the unique identifiers associated with the markers is discussed). 

Regarding claim 8, Dempski discloses the system is arranged so the virtual 
control panel displayed changes when another device is recognized and identified, and 
when the user has accepted the other device (see paragraph [0013] "selecting data 
from a memory storage with the data being associated with one of the objects 
associated with one of the visual markers, and then displaying the data on a wearable 
display worn by the operator."). 

Regarding claim 9, Foxlin discloses said portable identification element is 
adapted to be carried by the user during interaction with the virtual control panel (see 
paragraph [0058] where the wearable computer, 10 is discussed). 

Regarding claim 10, Foxlin discloses said portable identification element is 
attachable to a body of the user (see paragraph [0058] where the wearable computer, 
10 is discussed). 

Regarding claim 1 1 , Dempski discloses said display unit comprises a wearable 
display device showing the user said view (see paragraph [0008] where the wearable 
see-through display, 12 is discussed). 
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Claims 12 and 13 are similar in scope to claims 1 and 2, respectively, and are 
therefore rejected under similar rationale. 

Regarding claim 14, Foxlin discloses defining a two-way communication between 
the virtual control panel and the device, sending information to the device regarding the 
users actions with the virtual control panel, receiving data from the device, generating a 
graphical representation of the received data and displaying the data on the virtual 
control panel (see paragraph [0105] where selecting and manipulating objects is 
discussed). 

Regarding claim 15, Foxlin discloses said data is displayed on the virtual control 
panel in response to interactions between the user controlled pointing object and the 
virtual control panel (see paragraph [0057] and [0058] where the hand-mounted beacon 
14 is discussed). 

Regarding claim 16, Dempski discloses storing a plurality of pre-defined 
graphical interfaces, each representing a control panel of a particular device, 
determining which of the stored control panels to be displayed, and generating a 
graphical representation of the control panel to be displayed based on the pre-defined 
graphical interface of the control panel to be displayed (see Dempski, paragraph [0012] 
"The databases may also store data about the work environment, object details, current 
status and location of workers, etc."). 

Regarding claim 17, Dempski discloses at least one of the stored graphical 
interfaces comprises more than one graphical view to be displayed on the virtual control 
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panel, and which of the views to be displayed is determined based upon actions of the 
user (see paragraph [0022] "The longer that the operator gazes upon the object the 
system may optionally be programmed to provide additional information regarding the 
object."). 

Regarding claim 18, Dempski discloses recognizing and identifying a device, 
determining which of the stored control panels to be displayed based on the identified 
device, and generating graphical representation of the control panel of the identified 
device based on the stored graphical interface of the identified device and displaying a 
view comprising the real world and the virtual control panel of the identified device 
projected in a fixed relation to said portable identification element (see paragraph [0013] 
"selecting data from a memory storage with the data being associated with one of the 
objects associated with one of the visual markers, and then displaying the data on a 
wearable display worn by the operator."). 

Regarding claim 19, Dempski discloses each device is provided with a unique 
identification marking and a device is recognized by identifying the unique identification 
marking (see paragraph [0016] where the unique identifiers associated with the markers 
is discussed). 

Regarding claim 20, Dempski discloses the virtual control panel displayed is 
changed when another device is recognized and identified, and when the user has 
accepted the device (see paragraph [0013] "selecting data from a memory storage with 
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the data being associated with one of the objects associated with one of the visual 
markers, and then displaying the data on a wearable display worn by the operator."). 

Regarding claim 21, Foxlin discloses said portable identification element is 
carried by the user during interaction with the virtual control panel (see paragraph 
[0058] where the wearable computer, 10 is discussed). 

Regarding claim 22, Dempski discloses the virtual control panel comprises virtual 
interaction members and an audio and/or visual feedback is generated when the user 
activates any of the virtual interaction members (see paragraph [0022] "the system may 
be responsive to voice command of the operator"). 

Claim 23 is similar in scope to claim 1 and is therefore rejected under similar 
rationale. 



Response to Arguments 

5. Applicant's arguments filed 9/13/2010 have been fully considered but they are 
not persuasive. 

Regarding Applicant's arguments concerning the combination of Dempski and 
Foxlin failing to disclose an identification unit that defines a position and orientation of a 
virtual control panel, the Examiner respectfully disagrees. 

As noted in the previous Office Action, Dempski discloses a system for accessing 
and viewing data in a virtual environment (see paragraph [0013]). Foxlin discloses a 
head-mounted virtual reality system capable of tracking a hand-mounted object relative 
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to a user's head (see paragraph [0057] and [0058]). One cannot show nonobviousness 
by attacking references individually where the rejections are based on combinations of 
references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981 ); In re Merck & 
Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). 

Regarding Applicant's arguments concerning Foxlin failing to disclose a second 
tracking unit adapted to capture data representing a position of the portable 
identification element, the Examiner respectfully disagrees. 

Examiner notes that Foxlin's wearable computer 10 is connected to the 
orientation tracker 30 through cable 18 and thus inherently "captures data" of the 
computer. Examiner further notes that Foxlin's tracker can be "adapted" to capture any 
data. 

Regarding Applicant's arguments concerning the Examiner's assertions that 
Foxlin's tracker 30 reads on the claimed first and second trackers, the Examiner notes 
that although two structural elements are claimed, both functions of the respective 
elements are performed by Foxlin's tracker 30. 

Therefore, the rejection is maintained. 

Conclusion 

6. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
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TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to RASHAWN TILLERY whose telephone number is 571- 
272-6480. The examiner can normally be reached on M-F 8:30 AM - 4:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Dennis Chow can be reached on 571-272-4124. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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